Picture encoding with wavelet transform and block area weights

ABSTRACT

In wavelet transform encoding, high-quality encoding is to be realized by enabling picture quality control from one fractional area to another. An input picture  100  is read out in an amount corresponding to a number of lines required for wavelet transform and buffered in a memory unit  6 . The input picture then is wavelet transformed in a wavelet transform unit  2  and quantized in a coefficient quantizing unit  3 . In quantizing wavelet transform coefficients, the wavelet transform coefficients are multiplied by weighting coefficients from one sub-band to another. The weighting coefficients are determined using the analysis information of a specified block area in a picture, such as motion information and texture fineness information. This enables fine quantization control in terms of a picture block as a unit.

This application is a continuation of U.S. application No. 09/803,404,filed Mar. 9, 2001, now U.S. Pat. No. 7,016,546, the contents of whichare hereby incorporated in their entirety.

BACKGROUND OF THE INVENTION

1. Field of the Invention

This invention relates to a picture encoding method and apparatus forencoding a still image or moving pictures using wavelet transform.

2. Description of Related Art

Typical of the conventional representative picture compression systemsis the JPEG (Joint Photographic Coding Experts Group) systemstandardized by ISO (International Organization for Standardization).This JPEG system is a system for compressing and encoding mainly a stillpicture using DCT (discrete cosine transform). This JPEG is known togive acceptable encoded and decoded pictures on the condition that alarger number of bits is allocated. However, if, in this system, thenumber of encoding bits is diminished to a more or less extent, blockdistortion peculiar to DCT becomes outstanding to make subjectivedeterioration apparent.

Recently, such a system comprising splitting the frequency spectrum of apicture into plural bands by a set of filters combined from pluralhigh-pass and low-pass filters, known as a filter bank, and encoding thepicture from band to band, is being researched vividly. In particular,the wavelet encoding, which is free from a defect proper to DCT thatblock distortion becomes apparent at high compression, is thought to bepromising as a new technology to take the place of the DCT.

In encoding moving picture, there are currently known the MPEG-1, MPEG-2and the MPEG-4 of the MPEG (Moving Picture Image Coding Experets Group)system. Of these, the MPEG-2 is widely used for compressing theso-called DVD (Digital Versatile Disc). In the encoding techniques, usedin JPEG and MPEG, encoding control is made from one macroblock toanother. It is noted that several 8×8 blocks, each of which is aprocessing unit for DCT, usually 16×16 blocks, make up one macroblock.

At present, the JPEG system, the MPEG system or the so-called DV(digital video) system is used in a majority of video products, such aselectronic still cameras or video movies. These encoding techniques,exemplified by JPEG and MPEG, use DCT as the transform system. Since theabove products, employing the wavelet transform as basis, are predictedto be presented to the market, researches towards improving theefficiency of the encoding system are conducted briskly in many researchlaboratories. In actuality, the JPEG2000, a format expected as thenext-generation international standard system for a still picture, maybe said to be a successor to JPEG, and is being worked out byISO/IEC/JTC1SC29/WG1, which is of the same organization as that of JPEG.It is scheduled that recommendations for standardization of the part 1of the JPEG2000 will be issued in March 2001. In this JPEG 2000, it hasbeen decided that wavelet transform is to be used in place of the DCT ofthe existing JPEG as the transform system as the basis of picturecompression.

Meanwhile, if desired to acquire not only an encoded still picture ofhigh quality but also encoded high-quality moving pictures, it iscrucial to solve the following problems:

-   (i) Since the wavelet transform usually executes transform on the    entire picture, it is impossible to perform fine control for each    specified area in a picture, such as macro-block based control in    DCT of the MPEG or JPEG.-   (ii) For overcoming the drawback (i) above, there is such a    technique in which a picture is divided into tiles or blocks, each    being a rectangle, usually a square, of a specified size, and    encoding control is performed separately for each tile which is    regarded as a picture. The technique has a defect that, (a) if the    tile size is decreased, the encoding efficiency is lowered, and    that (b) if the compression ratio is raised, discontinuities between    neighboring tiles become outstanding to lower the subjective picture    quality significantly.-   (iii) In wavelet transform encoding, as in DCT encoding, picture    quality control is by quantization control. In general, if the    quantization step is increased, the number of bits generated is    suppressed, however, the picture quality is deteriorated.    Conversely, if the quantization step is lowered, the number of bits    generated is increased, however, the picture quality is improved.    This quantization control needs to be realized in terms of a    specified picture area as a unit, irrespective of whether or not the    control is to be used as encoding means for tile-based encoding as    in (ii) above.

SUMMARY OF THE INVENTION

It is therefore an object of the present invention to provide a methodand apparatus in which picture quality control from one partial area toanother, that has so far been retained to be difficult, can be realizedby a wavelet transform encoder to permit accurate picture qualitycontrol to improve the subjective picture quality as well as to allow tocope with both still and moving pictures by a sole encoder.

In one aspect, the present invention provides a picture encoding methodcomprising a storage step of writing and storing an input picture inmemory means from one line to another, a wavelet transform step ofapplying wavelet transform in the horizontal and vertical directionseach time a picture stored in the memory means reaches the number oflines required for wavelet transform, a quantization step of quantizingwavelet transform coefficients obtained from the wavelet transform step;and an entropy encoding step of entropy encoding quantized coefficientsfrom the quantization step when the number of samples of thequantization coefficients has reached the size required for entropyencoding. The quantization step quantizes the wavelet transformcoefficients, using at least one of weighting coefficients of a tableprovided at the outset for each sub-band generated on wavelet transformand weighting coefficients found from one block area picture forming apicture to another.

In another aspect, the present invention provides a picture encodingmethod comprising a storage step of writing and storing an input picturein memory means from one line to another, a wavelet transform step ofapplying wavelet transform in the horizontal and vertical directionseach time a picture stored in the memory means reaches the number oflines required for wavelet transform, a quantization step of quantizingwavelet transform coefficients obtained from the wavelet transform step,a block picture analysis step of analyzing the motion information in ablock picture and the degree of fineness of the texture for each blockarea in the input picture and an entropy encoding step of entropyencoding quantized coefficients from the quantization step when thenumber of samples of the quantization coefficients has reached the sizerequired for entropy encoding.

In still another aspect, the present invention provides a pictureencoding method comprising a storage step of writing and storing aninput picture in memory means from one line to another, a wavelettransform step of applying wavelet transform in the horizontal andvertical directions each time a picture stored in the memory meansreaches the number of lines required for wavelet transform, aquantization step of quantizing wavelet transform coefficients obtainedfrom the wavelet transform step and an entropy encoding step ofresolving quantization coefficients obtained from the quantization stepinto bit planes from MSB to LSB, shifting bit planes of a plurality ofblocks present in the same sub-band by a pre-set number of bits, andentropy encoding the bit planes of sequentially entropy encoding blockswhen the number of samples of the shifted quantization coefficients hasreached a pre-set magnitude.

In yet another aspect, the present invention provides a picture encodingmethod comprising a storage step of writing and storing an input picturein memory means from one line to another, a wavelet transform step ofapplying wavelet transform in the horizontal and vertical directionseach time a picture stored in the memory means reaches the number oflines required for wavelet transform, a quantization step of quantizingwavelet transform coefficients obtained from the wavelet transform stepand an entropy encoding step of resolving quantization coefficients fromthe quantization step into bit planes from the MSB to the LSB and forentropy encoding the quantization coefficients when the number ofsamples of the quantization coefficients has reached a pre-setmagnitude. The entropy encoding step splits and extracts fractionalportions of the bit planes from the MSB to the LSB of a plurality ofentropy encoding blocks existing in one and the same sub-band, encodesthe extracted fractional bit planes from the MSB to the LSB, andgenerates the fractional encoded bitstream corresponding to thefractional bit planes.

That is, according to the present invention, a spatial block forming apicture is wavelet transformed, responsive to the analysis informationfor the spatial block, to generate coefficients in a sub-band, whichthen are acted on to enable optimum quantizing control. In this manner,picture quality control, retained to be difficult, can be realized by awavelet transform encoding apparatus.

Moreover, according to the present invention, the quantizationcoefficients are multiplied by weighting coefficients provided at theoutset for respective sub-bands, the respective quantizationcoefficients are resolved into bit planes, and the sequence of encodingthe bit planes from the MSB to the LSB is made variable from one entropyencoding block to another to realize quantization control to enablepicture quality control accurately.

In addition, the present invention can be applied to moving pictureencoding apparatus adapted for coping with plural frames, so that aninexpensive apparatus may be provided which is able to cope with bothstill and moving pictures with a sole encoding unit.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram showing a schematic structure of a pictureencoder as a first embodiment of the present invention.

FIG. 2 illustrates the operation of wavelet transform encoding.

FIG. 3, in continuation to FIG. 2, illustrates the operation of wavelettransform encoding.

FIG. 4, in continuation to FIG. 3, illustrates the operation of wavelettransform encoding.

FIG. 5, in continuation to FIG. 4, illustrates the operation of wavelettransform encoding.

FIG. 6 illustrates the band splitting of a two-dimensional picture, withthe splitting level being 3.

FIG. 7 is a schematic block diagram showing a specified illustrativestructure of a coefficient quantizer 3 of FIG. 1.

FIG. 8 is a schematic block diagram showing another specifiedillustrative structure of a coefficient quantizer 3 of FIG. 1.

FIG. 9 is a block diagram showing areas of pre-set spatial pictureblocks from one sub-band to another.

FIGS. 10 a, 10 b and 10 c illustrate coefficients extended in a bitplane.

FIG. 11 is a schematic view block diagram showing a schematic structureof a picture encoder as a third embodiment of the present invention.

FIG. 12 is a schematic view block diagram showing a schematic structureof a picture encoder as a fourth embodiment of the present invention.

FIGS. 13A, 13B and 13C illustrate the concept of tile-based wavelettransform encoding.

FIG. 14 illustrates convolutional processing in overlapped waveletencoding.

FIG. 15 illustrates the concept of symmetrical convolutional processing.

FIGS. 16A, 16B, 16C, 16D, 16E, and 16F illustrate the concept of waveletencoding performing symmetrical convolutional pixel processing.

FIGS. 17 a and 17 b illustrate the concept of point-symmetricalconvolutional processing.

FIGS. 18A and 18B illustrate a sub-band obtained on wavelet splitting ofa picture and the corresponding bit plane.

FIGS. 19A and 19B illustrate bit plane shift processing of a blockobtained on sub-band splitting a specified picture area.

FIG. 20 illustrates bit plane encoding of an entropy encoded blockpresent in a sub-band.

FIG. 21 illustrates frame and field pictures for an interlaced picture.

FIG. 22 illustrates a bit plane and a sub-bit plane.

FIG. 23 illustrates sub-band areas generated following wavelettransform.

DESCRIPTION OF THE PREFERRED EMBODIMENTS

Referring to the drawings, preferred embodiments of the presentinvention will be explained in detail.

FIG. 1 is a block diagram showing an illustrative structure of a pictureencoding device according to a first embodiment of the presentinvention.

A picture encoding device, shown in FIG. 1, includes a memory 6 forwiring input picture data (input picture) 100 line-by-line for writingand storage therein, a wavelet transform unit 2 for applying wavelettransform in the horizontal and vertical directions each time a picturestored in the memory 6 reaches the number of lines required for wavelettransform, and a coefficient quantizing unit 3 for quantizing wavelettransform coefficients obtained from the wavelet transform unit 2. Thepicture encoding device also includes an entropy encoding unit 4 forapplying entropy encoding when the number of samples of the quantizationcoefficients obtained from the coefficient quantizing unit 3 has reacheda size required for entropy encoding. The coefficient quantizing unit 3performs quantization using one or both of the weighting coefficients ofa table provided at the outset for each sub-band generated at the timeof the wavelet transform and weighting coefficients found from one blockarea picture forming a picture to another. It is noted that the inputpicture 100 is also fed to a block picture analysis unit 1 adapted foranalyzing the motion information in a block picture or the fineness ofthe texture from one block area picture forming a picture to another.This block picture analysis unit 1 will be explained subsequently indetail.

In the picture encoding device, shown in FIG. 1, an input picture 100 iscaptured sequentially on the line basis, beginning from the uppermostline, to input the captured picture to a data readout memory 6. At atime point a pre-set amount of picture data corresponding to the numberof lines required for wavelet transform has been stored in the memory 6,the wavelet transform unit 2 applies wavelet transform filtering in thehorizontal and vertical directions. Usually, the filter used forfiltering in the wavelet transform is a filter having plural taps. Ifthe number of lines required for wavelet transform is stored, thewavelet transform filtering can be executed immediately.

FIGS. 2 to 5 illustrate specified processing for wavelet transform,wavelet splitting processing and entropy encoding. The input picture isread and stored from one data line 21 to another, in the memory 6 ofFIG. 1 (buffer 22 of FIG. 2), as indicated at step S1. If the number oflines required for vertical filtering in the wavelet transform isstored, as indicated at step S2, the vertical filtering is performed inthe wavelet transform unit 2 of FIG. 1. Then, horizontal filtering isperformed, as indicated at step S3 in FIG. 3. The wavelet transformcoefficients, obtained on wavelet transform, are stored in the buffer11. At this time point, the values of wavelet transform coefficients offour sub-bands (LL, LH, HL, HH) are already determined. The coefficientquantizing unit 3 of FIG. 1 executes quantization on sub-bandcoefficients 23 of three sub-bands LH, HL and HH on the high frequencyside, shown shaded, as indicated at step S4. At this time, a controlsignal 122 for executing the quantization is sent from the controller 5to the coefficient quantizing unit 3. The respective sub-bands (LL, LH,HL and HH) will be explained subsequently with reference to FIG. 6.

On the other hand, wavelet transform coefficients 24 of the lowestsub-band (LL) are again stored in the buffer. This processing iscontinued until the number of lines as necessary for vertical filteringis stored, as indicated at step S5. So, wavelet transform coefficients120 from the wavelet transform unit 2 are sent line-by-line to andstored in the buffer 11. On the other hand, quantization coefficients103 (data of quantized sub-band coefficient lines 25 of FIG. 3),obtained on quantization by the coefficient quantizing unit 3 at stepS4, are sent to the entropy encoding unit 4.

If the number of lines required for vertical filtering has been storedin the buffer 11 for the lowest sub-band LL, a control signal 105 forexecuting the wavelet transform is sent from the controller 5 to thewavelet transform unit 2. For generating the next waveform splittingstage, horizontal filtering is executed next to the vertical filtering,as shown at step S6. As a result, the wavelet transform coefficientsvalues of four sub-bands of the second stage of the lowest sub-band arehere determined, as indicated at step S7 in FIG. 4. So, quantization ofthe subsequent stage is performed immediately to output quantizationcoefficients.

If the number of lines required for vertical filtering for wavelettransform in the processing at step S2 of FIG. 2 is to be stored, for acase where the number of split stages is one, or if the number of linesrequired for vertical filtering in the processing at step S5 of FIG. 3is to be stored, for a case where the number of split stages is two, thewavelet transform coefficients are stored and held in the buffer 11 ofFIG. 1. At this time, the wavelet transform coefficients 120 in eachsplit stage are sequentially sent to and stored in the buffer 11 on theline basis.

On the other hand, in case of vertical filtering in the processing ofstep S33 or in the processing of step S6 in FIG. 3, the wavelettransform coefficients 121 11 are read out from the buffer 11 in anamount corresponding to the required number of lines stored in thebuffer, and are subjected to vertical filtering. The above-describedsequence of operations is continued until the processing for thetotality of splitting stages come to a close. It is however possible toperform entropy encoding, as now explained, on quantized coefficientsfor which entropy encoding has become possible partway.

That is, the entropy encoding by the entropy encoding unit 4, located atthe trailing end of FIG. 1, is also generally termed an informationsource encoding, and is a technique for compressing the informationvolume by exploiting such properties as offset in the distribution ofoccurrence of data strings. For this entropy encoding, the Huffmanencoding and arithmetic coding are used extensively. It is noted thatmeans for performing the encoding as a data string is input and learnedis likely to be more favorable in adaptability to the input data thanthe Huffman encoding employing a pre-set table. In this case, the rangein which the input data is to be acquired represents a problem. Ingeneral, a larger amount of the input data is more meritorious. However,if desired to limit the processing to a specified area in a picture,input data is acquired within a range of a block-shaped picture of acertain size.

In the present embodiment, any of the above cases can be coped with.However, if entropy encoding is to be performed after holdingquantization coefficients of the entire picture, a large size memory orbuffer sufficient to hold the entire coefficients is required. So, theentropy encoding of a block base for which a small memory capacitysuffices is hereinafter explained.

When the quantization of the wavelet transform coefficients proceeds bystep S4 of FIG. 3 and by step S7 of FIG. 4, until the number of lines ofthe quantization coefficients of the sub-bands (HL, LH, HH) of the firstsplit stage reaches the height H of the block of the block-based entropyencoding processing unit, the control signal 107 for executing theentropy encoding is sent from the controller 5 to the entropy encodingunit 4 where entropy encoding is executed from one entropy encoding unit28 to another.

In similar manner, when the number of lines of the quantizationcoefficients of the sub-band LL of the second splittings stage hasreached the height H of the block of the entropy encoding processingunit for block-based entropy encoding, as indicated at step S9 of FIG.5, entropy encoding is executed from one entropy encoding unit 28 toanother.

Meanwhile, the block 29 of FIG. 5 indicates a block of quantizationcoefficients from entropy encoding.

The above processing is repeated up to the wavelet splitting stage asrequired to complete wavelet transform and quantization plus entropyencoding of the entire picture.

In the first embodiment of the present invention, having the structureand the operation as described above, quantization is executed, usingthe weighting coefficients provided at the outset from one sub-bandgenerated in the wavelet transform processing to another and/or theweighting coefficients as found from one block area picture forming anentire picture to another. The specified structure and operation of thecoefficient quantizing unit 3 of FIG. 1 are hereinafter explained.

In a usual wavelet transform encoder, wavelet transform coefficients aredirectly quantized, however, the quantized coefficients are multipliedwith weighting coefficients to correct the values of the quantizedcoefficients.

For example, there is shown two-dimensional wavelet transform in FIG. 6,where band components obtained as a result of band splitting of atwo-dimensional picture up to level 3 are shown. That is, fourcomponents LL, LH, HL and HH are obtained by level 1 band splitting inthe horizontal and vertical directions, where LL denotes both horizontaland vertical components being L and LH denotes the horizontal andvertical components being L and H, respectively. The LL component isagain band-split to form LLLL, LLHL, LLLH and LLHH, whilst the LLLLcomponentis further band-split to form LLLLLL, LLLLHL, LLLLLH andLLLLHH. Instead of hierarchically splitting the band in this manner, theentire band may be split equally. It may be seen that, in the embodimentof FIG. 6, there is obtained a sum total of ten sub-bands by waveletsplitting up to the third level. These ten sub-bands are multiplied bye.g., weighting coefficients T. The weighting coefficients T forrespective sub-bands are T_(LLLLLL), T_(LLLLHL), T_(LLLLLH), T_(LLLLHH),T_(LLHL), T_(LLLH), T_(LLHH), T_(LH), T_(HL) and T_(HH), beginning fromthe coefficient for the low range side LLLLLL component.

That is, the totality of the transform coefficients of the HH componentas the level 1 high range band are multiplied by the value of thecoefficient T_(HH), Similarly, the transform coefficients of the otherranges of the level 1 are multiplied by the coefficients T_(LH), T_(HL).The LL component is further split into four bands by the splitting ofthe level 1, so that, in similar manner, the respective transformcoefficients are multiplied by weighting coefficients which arepredetermined for the respective bands. The above-described operation isexecuted repeatedly up to a pre-set wavelet splitting level to correctthe transform coefficients. In this case, the weighting coefficients maybe of different values from one sub-band to another.

Specifically, larger values of the transform coefficients are desirablyused as the band goes to lower sides, such as T_(LLLLLL) or T_(LLHL).

The technique of managing fine control by multiplying the quantizationcoefficients with weighting coefficients from one specified block areaof the image space to another is now explained. FIGS. 7 and 8 illustratea specified illustrative structure of the coefficient quantizing unit 3of FIG. 1.

The coefficient quantizing unit of FIG. 7 includes a scalar quantizingunit 13, fed with wavelet transform coefficients 102 from the wavelettransform unit 2 of FIG. 1, a block area weighting coefficient changingunit 14, fed with scalar-quantized quantization coefficients 124, and asub-band based weighting coefficient table 15 for routing the sub-bandbased weighting coefficients 125 to the block area weighting coefficientchanging unit 14. This block area weighting coefficient changing unitissues ultimate quantization coefficients 103, as outputs, to theentropy encoding unit 4 of FIG. 1. On the other hand, the analysisinformation 106, specifying the block area the picture quality for whichis desired to be raised, is routed by the block picture analysis unit 1to the block area weighting coefficient changing unit 14. This blockpicture analysis will be explained in detail subsequently.

That is, the wavelet transform coefficients 102, generated by thewavelet transform unit 2 of FIG. 1, as described above, are quantized bythe scalar quantizing unit 13 as illustrative quantization means, togenerate quantization coefficients 124, which are routed to the blockarea weighting coefficient changing unit 14. On the other hand, thesub-band based weighting coefficient table 15 issues sub-band basedweighting coefficients 125, as explained with reference to FIG. 6. Theseweighting coefficients are set so as to be larger or smaller for largeror smaller values of the numbers of wavelet splitting stages,respectively. On the other band, the block area weighting coefficientchanging unit 14, fed with the block area analysis information, computethe weighting coefficient values for a block area in question to outputthe changed quantization coefficients 103 to the block area weightingcoefficient changing unit 14. As a specified operation, if it is desiredto raise the picture quality of a desired block area, the weightingcoefficients need to be set to higher values. So, the coefficients ofthe portion of the sub-band corresponding to such block area are set tolarger values based on the sub-band based weighting coefficients 125.

Assume that the areas corresponding to center blocks, shown netted inFIG. 9, of nine blocks obtained on dividing a picture by 3 in thehorizontal direction and by 3 in the vertical direction, is theabove-mentioned area desired to be raised in picture quality, as shownin FIG. 9. If this portion is to be raised in picture quality, the ruleof using the same weighting coefficient T within one and the samesub-band as shown in FIG. 6 is discounted and, based on this T, thevalues of the weighting coefficients for the relevant areas (nettedportions) are changed. For example, the value of the weightingcoefficient for the sub-band component HL is T_(HL), as indicated inFIG. 6. By multiplying the transform coefficient for an area P₁ inquestion in the sub-band component with a weighting coefficient valueT_(HL) for the sub-band HL larger than the value for other than the areaP₁, the weighting coefficient value of the area P₁ in the sub-bandcomponent HL can be set so as to be larger than that for the other area,so that the picture quality this block area may be improved. The areasP₁, P₂ and POSITIVE ELECTRODE 3 of the other sub-bands can be processedin an identical manner.

By multiplying the weight value of the block area with a weightingcoefficient of the sub-band component, the weighting coefficient of theblock area may be determined and multiplied to the scalar quantizationcoefficient sent from the scalar quantizing unit 13 to determine theultimate quantization coefficient 103 which is then issued as an output.FIG. 8 shows the structure of this specified embodiment. Referring toFIG. 8, the block area analysis information 106 is routed to the blockarea weighting coefficient computing unit 16 to find a block area weight123 which is routed to a block area weight coefficient computing unit17. The block area weight coefficient computing unit then multiplies theblock area weight 123 with the value of the weighting coefficient 125 ofthe sub-band component from the sub-band based weighting coefficienttable 15 to find the weighting coefficient of the block area inquestion. This weighting coefficient is multiplied to the scalarquantization coefficient 124 sent from the scalar quantizing unit 13 tooutput an ultimate quantization coefficient 103.

The scalar quantizing unit 13 executes the scalar quantization as shownfor example by the following equation 1:Q=x/Δ  (1)to give a scalar quantization output where x and Δ denote wavelettransform coefficient and the quantization index value, respectively.

The block picture analysis unit 1 and the block area analysisinformation 106 of FIG. 1 is now explained. The block picture analysisunit 1 of FIG. 1 extracts the information that, in e.g., a block areapicture, there is a marked movement of an object or the texture ishighly detailed, by analysis means adapted for analyzing the motioninformation or fineness of the texture from one block area picture toanother. Specifically, the results of search in the field of pictureprocessing may be used. In the motion detection, for example, adifference between the current frame and a directly previous frame isfound and, if the difference is larger than a pre-set threshold value,it is decided that there is a motion. As for texture fineness, thevariance values of the totality of pixel values in a block area pictureare sampled and, if the sampled value is larger than a pre-set thresholdvalue, the texture is verified to be detailed.

In the above-described first embodiment, wavelet transform is executedin the order of a vertical filtering and horizontal filtering. Thisorder may, of course, be reversed, provided that the data bufferingdirection for the horizontal direction is naturally reversed from thatfor the vertical direction.

Second Embodiment

A second embodiment of the present invention is now explained. In thissecond embodiment, the entropy encoding unit 4 shown in FIG. 1 isconstructed so that the quantization coefficients in a block arearranged into a bit plane composed of binary data, arithmetic encodingis executed depending upon the occurrence probability distribution ofthe symbols of each sub-bit plane and the probability distribution isestimated only for data in a pre-set block.

Referring to FIG. 10, the bit plane is explained with reference to FIGS.10 a to 10 c. FIG. 10 a shows 4 vertical by 4 horizontal or 16quantization coefficients, with +13, −6 and so forth indicatingpost-quantization coefficient values. These quantization coefficientsare divided into absolute values and signs of plus and minus, with theabsolute values being expanded into a bit plane from the MSB to the LSB.FIGS. 10 b and 10 c indicate bit planes of the absolute values and a bitplane of signs, respectively. The coefficients on each absolute valuebit plane of FIG. 10 b are 0 or 1, whilst the coefficients on the signbit plane in FIG. 10 c are +, 0 and −. In FIGS. 10 b and 10 c, there areshown four absolute value bit planes and one sign bit plane. So, by wayof the post-stage processing, it suffices to carry out bit plane basedbi-level encoding, such as arithmetic coding. The foregoing is theoperation of the entropy encoding unit 4 of FIG. 1.

Meanwhile, the arithmetic coding is the technique of fitting numericalvalues to encoding symbols as probability distribution estimation iscarried out within a pre-set range. This probability distributionestimation is carried out within a range of a pre-set block, asexplained with reference to FIG. 4. This maintains independence ofentropy encoding.

Third Embodiment

A picture encoding device, as a third embodiment of the presentinvention, is explained with reference to FIG. 11. The picture encodingdevice, shown in FIG. 11, is made up of a full picture memory unit 18for storing the full picture of the input image, a full picture wavelettransform unit 7 for applying the wavelet transform filtering in thehorizontal and vertical directions to the entire picture, a blockcoefficient extraction unit 8 for extracting wavelet transformcoefficients corresponding to a specified block area forming a picture,a block coefficient quantizing unit 9 for quantizing the extractedwavelet transform coefficients, and an entropy encoding unit 4 forexecuting entropy encoding when the number of samples of generatedquantization coefficients has reached a pre-set magnitude.

The first embodiment, already explained, differs from the thirdembodiment shown in FIG. 11 in that the first embodiment executeswavelet transform line by line, whereas the third embodiment shown inFIG. 11 performs wavelet transform of the entire picture once andperforms subsequent processing, such as quantization and entropyencoding, on the block basis.

In FIG. 11, the entire input picture is input to and stored in thememory 18. The full picture wavelet transform unit 7 then performshorizontal and vertical filtering for wavelet transform on the entirepicture data 108. Transform is carried out up to the pre-set number ofwavelet splitting to generate transform coefficients 126 of the totalityof sub-bands. A transform coefficient buffer unit 19 plays the role of abuffer for primarily storing and holding the transform coefficients 126.

In accordance with the control signal 112 from the controller 5, theblock coefficient extraction unit 8 extracts and reads out, from thetotality of the transform coefficients stored in the transformcoefficient buffer unit 19, the transform coefficients 109 of thetotality of the sub-bands corresponding to the block area being encoded.The sub-bands of partial areas, shown netted in the drawings, arealready explained with reference to FIG. 9. The transform coefficients110 from the block coefficient extraction unit 8 are then quantized inthe block coefficient quantizing unit 9.

As already explained in the first embodiment, the block picture analysisunit 1 sends the analysis information 106 comprehending the motion andtexture information for each rectangular block area picture forming afull picture. The block coefficient quantizing unit 9 is responsivethereto to have reference to the analysis information 106 of the currentblock to be encoded to execute quantization from one block to another.This specified operation has already been explained in the firstembodiment with reference to FIGS. 7 and 8. If it is desired to raisethe picture quality of a pre-set block picture, it suffices if theweighting coefficients are set to larger values, the sub-band basedweighting coefficient table values as explained with reference to FIG. 6are computed from the so-set values and the resulting computed valuesare multiplied with the scalar quantization coefficients.

Since the foregoing enables detailed quantization control, from oneencoding block to another, the adaptive picture quality control can berealized with advantage.

The quantization coefficients 111 of the block, ultimately obtained bythe above processing, are entropy encoded in the entropy encoding unit4, in accordance with the control signal 5, to produce an encodedbitstream 113, which is issued as output. The entropy encoding employingthe bit plane as explained in connection with the second embodiment mayalso be applied.

In this third embodiment, a picture is buffered and wavelet transformedonce and processed with quantization and entropy encoding on the blockbasis in the subsequent stage, so that picture readout can be done onlyonce with advantage as compared to the case of the first embodiment.However, in the present third embodiment, the buffer capacity asrequired is increased.

Fourth Embodiment

The fourth embodiment of the present invention includes tile splittingmeans, upstream of the wavelet transform encoding unit, for splittingthe input picture into plural rectangular tiles, and a downstream sideencoding means for encoding picture data in each tile picture read outinto a memory. An illustrative structure of the picture encoding deviceof the present fourth embodiment is shown in FIG. 12. The pictureencoding device, shown in FIG. 12, is similar to the encoding device ofFIG. 1 except that a tile splitting unit 10 is provided on the upstreamside of the picture encoding device shown in FIG. 1. So, the encodingdevice, shown in FIG. 12, is not explained specifically.

FIG. 13 illustrates the operation of splitting the original picture intoplural tiles and applying wavelet transform to each tile. In FIG. 13,solid lines indicate the boundaries of actual tiles and broken linesindicate the boundary of an area affected by wavelet transform filteringas later explained.

That is, the original picture, shown in FIG. 13A, is split into pluraltiles, as shown in FIG. 13B. Each tile is subjected to wavelet transformfiltering in a range up to the boundary shown by a broken line in FIG.13C to generate an encoded bitstream.

In FIG. 12, each tile picture 114 from the tile splitting unit 10 isinput to the block picture analysis unit 1 where the analysisinformation 115 is produced by the technique as described above and isissued as output. It should be noted in this connection that the tilepicture 114 need not necessarily be the same as the aforementioned blockpicture. That is, plural blocks may be present in one tile. In general,the tile size is set so as to be larger than the block size. However,for simplifying the processing, such as encoding, these sizes areroutinely set to as to be powers of 2.

A tile picture 114 from the tile splitting unit 10 of FIG. 12 is storedand held in the memory 6. A explained in connection with the firstembodiment, when the picture data are read out from line to line,buffered and wavelet transformed, the tile picture 114 is read out lineby line and stored in the memory 6. On the other hand, if the entirepicture is to be buffered once and for all as described above inconnection with the third embodiment, the totality of picture data inthe tile picture 114 are stored in the memory 6. The present fourthembodiment is able to cope with any of these configurations. Theoperation downstream of the wavelet transform unit 11 has already beexplained in connection with the respective embodiments.

Picture data 116, sent out from the memory 6, are wavelet transformed inthe wavelet transform unit 11 in accordance with the control signal fromthe control unit 5 and wavelet transformed to output transformcoefficients 117. These transform coefficients 117 are quantized in thecoefficient quantizing unit 3 to produce quantization coefficients whichare routed to the entropy encoding unit 4. The entropy encoding unitperforms entropy encoding, based on a control signal 105 from thecontrol unit 5, so that an encoded bitstream 119 is sent out by theentropy encoding 4. The operation of adaptively quantizing thecoefficients using the analysis information 115 from the block pictureanalysis unit 1 is similar to that already explained in connection withthe previous embodiments.

Fifth Embodiment

This fifth embodiment achieves the filtering operation up to neighboringtiles in the wavelet transform of tile pictures in the above-describedfourth embodiment.

If wavelet transform is applied from tile to tile, as described above,it is necessary to take account of tiles other than the tiles beingencoded. That is, if wavelet transform is performed from tile to tile,the filtering affects the pixels around the tile by a lengthcorresponding to the tap length of the filter. So, the filtering isperformed with overlap with respect to neighboring tiles. This overlaptype tile-based wavelet transform, in which pixels of neighboring tilesaffected by the filtering are read out for wavelet transform, is nowexplained with reference to FIG. 14.

FIG. 14 shows tile or blocks R_(T) to be encoded and a range R_(F)affected by filtering in case of performing the tile-based wavelettransform. In FIG. 14, a to f and h to m all denote pixels. For example,if the pixel c is to be filtered in the horizontal direction, threepixels d, e, f are read out from the right neighboring tile picture andconvolved with pre-set filtering coefficients. In similar manner, if thepixel j is to be filtered in the vertical direction, three pixels k, land m are read out from the lower tile picture and convolved withpre-set filtering coefficients.

Thus, in the tile splitting unit of the present fifth embodiment, aframe picture is split into plural tiles, whilst pixels of neighboringtiles, affected by the filtering by the wavelet transform means, areread out to enlarge the area being encoded. In this case, the manner ofacquiring the pixels of the area affected by the filtering outside thetile being encoded represents a problem.

In this fifth embodiment, there is provided no overlapping area betweenpictures of neighboring tiles. Instead, the wavelet transformcoefficients within the tile are symmetrically extended and processed byconvolution within an area outside tile affected by the filtering. Thisis shown in FIG. 15 specifically showing an area around a tile and inFIG. 16 showing the process in which wavelet splitting is performed asconvolution processing is performed on the original picture forsymmetrical expansion.

Referring to FIG. 15, illustrating this symmetrical expansion, it willbe seen that a horizontal pixel array of c, b, a in a tile area R_(T) tobe encoded or decoded, is expanded in the arraying order of a, b, c,symmetrically with the tile boundary as a boundary, up to R_(F)delimiting an area affected by the filtering. In the vertical direction,a pixel array f, e, d in the tile area RT is extended in the arrayingorder of d, e, f, symmetrically with the tile boundary as a boundary, upto R_(F) delimiting an area affected by the filtering. It has been knownthat, by this mirror-image symmetrical expansion, only the same numberof wavelet transform coefficients as the number of pixels in the tilepicture is generated, thus eliminating the redundancy.

The concept of wavelet encoding by symmetrical convolution is shown inFIGS. 16A to 16C. The original picture, shown in FIG. 16A, is split intotile pictures, shown in FIG. 16B. For each tile picture, pixels aresymmetrically expanded to an area outside the tile, for each tilepicture, up to a broken line in FIG. 16C delimiting an area affected byfiltering.

The wavelet transform (WT) then is applied to each tile, symmetricallyexpanded as shown in FIG. 16C. As a result, each tile picture is splitinto e.g., four band components, as shown in FIG. 16D. The areas shownshaded in FIG. 16D represent the low-range components LL. Each tile ofthe low range component LL, shown shaded, is expanded symmetrically, insimilar manner, as shown in FIGS. 16E and 16F, by way of executing thewavelet transform (WT). The similar symmetrical expansion is performeduntil a pre-set number of wavelet splitting is reached. The foregoing isthe explanation of the wavelet transform with tile-based symmetricalexpansion in the tile splitting and wavelet transform areas of the fifthembodiment.

Sixth Embodiment

In a sixth embodiment of the present invention pixels for encodingoutside the tile are expanded by symmetrical expansion, at the time oftile-based wavelet transform, so that the pixels in an area outside thetile affected by filtering are symmetrically expanded in apoint-symmetrical relation with respect to the pixel value on the tileboundary.

FIGS. 17 a and 17 b show two examples in which eight sample points X[0],X[1], X[2], X[3], X[4], X[5], X[6], X[7] denote pixels in the tile (onlyin a one-dimensional direction).

On the other hand, pixels of sample points, indicated by broken lines,are computed with the pixel values of X[0] or [7] as reference valuesfor point symmetry. In FIG. 17 a, a point Pa denoting the pixel value ata sample position of a sample point X[0] on the tile boundary is areference point of point symmetry. For example, a sample point x[1]outside the tile is obtained by calculating an equidistant point on astraight line extended from the sample point X[1] at a point-symmetricalposition in the tile through the reference point Pa. That is, the samplepoint x[1] is at a position of point symmetry with respect to the samplepoint X[1]. In similar manner, sample point [x]2 and x[3] are atpoint-symmetrical positions with respect to the sample points X[2] andX[3], with the reference point Pa as center. The same applies for theopposite side boundary of the tile where the pixel value at a samplepoint X[7] is to be a reference point of point symmetry.

The case of FIG. 17 b differs from that of FIG. 17 a in that a referencepoint Pb as the center of point symmetry is offset a distance equal to ahalf sample with respect to a sample position. That is, a point Pbdenoting a pixel value equal to a sample point X[0] at a position offseta distance equal to one half sample to outside the tile from a sampleposition of the sample point X[0] at the tile boundary is a referencepoint of point symmetry. Thus, a sample point x[0] at a position spacedone sample to outside the tile from the sample point X[0] is at a pointsymmetry with respect to the sample point X[0] with the reference pointPb as center, so that it is of the same value as that of the samplepoint X[0]. In this manner, a point on the inner side of the tileboundary, such as X[0], and a point outside the tile boundary, such asx[0], are of the same magnitude, thereby assuring smooth junction on thetile boundary.

By pixel expansion employing the relation of point symmetry by any ofthe techniques shown in FIG. 17 a or 17 b, the pixel values in the tilecan be enlarged to an area affected by filtering by wavelet transformoutside the tile, without employing the pixel values of neighboringtiles, by way of performing convolution processing.

Seventh Embodiment

A seventh embodiment of the present invention is explained. In thisseventh embodiment, only a part of pixels within an area outside thetile affected by filtering are extracted for convolution processing.

In the above-described fifth and sixth embodiments, the totality ofpixels within an area affected by filtering are found. In the presentseventh embodiment, only a part of these pixels are extracted. Byextracting only a part of the pixels within the area affected byfiltering, the number of times of convolution processing can beadvantageously reduced. Although picture quality deterioration(discontinuities) are likely to be produced at the tile boundary, nosignificant difference is produced in case of using a higher encodingbitrate.

Eighth Embodiment

The present eighth embodiment proposes another configuration of theabove-described block-based quantization control. Although the samestructure as that explained with reference to FIG. 1 may be used, theoperation in the coefficient quantizing unit 3 differs. Here,quantization coefficients obtained on scalar quantization of wavelettransform coefficients, generated by the wavelet transform unit 2, aspreviously explained, are resolved into bit planes from the MSB to theLSB. The concept of the bit planes has been explained with reference toFIG. 10 and therefore is not explained for simplicity.

As the operation occurring in the coefficient quantizing unit 3, binarydata are entropy encoded, in a usual case, from one bit plane toanother, in the direction from the MSB to the LSB of the bit plane, asexplained in connection with the above-described second embodiment.However, in the present eighth embodiment, the bit planes of the blocks,obtained on splitting into plural sub-bands present in the samesub-band, are shifted in an amount corresponding to a predeterminednumber of bits and newly generated bit planes are entropy encoded.

The present embodiment is now explained with reference to FIGS. 18A and18B. FIG. 18A shows a sub-band obtained on wavelet splitting a picture.6 vertical by 6 horizontal blocks, totalling at 36 blocks, as divided bybroken lines, are assumed to be block units for downstream side entropyencoding. FIG. 18B is a cross-sectional view of a bit plane in asub-band. Although the bit plane is two-dimensional, it is shown asbeing one-dimensional. In FIG. 18B, the width of the entropy encodingunit is denoted W. The netted area in a sub-band of FIG. 18A correspondsto the bit plane shown shaded in FIG. 18B, in which the ordinaterepresents the depth of the bit plane from the MSB to the LSB.

Meanwhile, one block area in an original picture has its vertical andhorizontal sizes halved as the splitting stage is incremented by one, asexplained with reference to FIG. 6. That is, in the block area P (nettedarea) of FIG. 9, one-half the vertical and horizontal sizes of thefirst-level block area P₁ becomes the vertical and horizontal sizes ofthe second-level block area P₂. In general,

-   -   (vertical and horizontal size of P_(n+1))=(vertical and        horizontal size of P_(n))/2.        So, the relative magnitudes of a block of a certain sub-band,        obtained on wavelet splitting a specified area of a picture, and        the above-described entropy encoded block, need to be taken into        consideration.

If the size W_(S) of a sub-band block of a specified area of a picture,shown netted in FIG. 19A, is smaller than the size W_(E) of an entropyencoded block, that is if W_(S)<W_(E), as shown in FIG. 19A, the bitplane corresponding to the sub-band of a specified area of a picture isshifted up towards the MSB side, or shifted down towards the LSB side.After the shifting operation of the totality of sub-band blocks in theentropy encoding block is finished, entropy encoding processing isperformed on the newly generated bit plane. Meanwhile, the shift-upoperation contributes to improved picture quality of the block area,whereas the shift-down operation deteriorates the picture quality. Thisachieves picture quality control.

The bit plane shifting-down operation may also be performed on theentropy encoding blocks other than those existing in the block area inquestion to achieve equivalent effects as those described above. On thedecoder side, a reverse operation to this shifting operation isrequired. This bit shifting value needs to be written in the encodedbitstream.

On the other hand, FIG. 19B shows a case wherein the size W_(S) of thesub-band block of a specified area of a picture is larger than or equalto the entropy encoding block size W_(E), that is a case whereinW_(S)≧W_(E). In this case, the totality of the entropy encoding unitspresent in the sub-band split block is bit-plane encoded after theshifting operation of the sub-band split block is finished.

Ninth Embodiment

The picture encoding device according to a ninth embodiment of thepresent invention writes an input picture on the line basis in memorymeans for storage, applies wavelet transform in the horizontal andvertical directions each time a picture stored in memory means reachesthe number of lines required for wavelet transform, quantizes thewavelet transform coefficients obtained on wavelet transform, resolvesthe produced quantization coefficients into bit planes from the MSB tothe LSB and performs entropy encoding when the number of samples of thequantization coefficients reaches a pre-set size. In performing theentropy encoding, a portion of the bit plane from the MSB to the LSB issplit and extracted in each of the bit planes of plural entropy encodingblocks present in one and the same sub-band, and the fractional bitplanes are encoded from the MSB to the LSB to generate the fractionalbitstreams corresponding to the fractional bit planes.

In the ninth embodiment, the operation until resolution of thequantization coefficients into bit planes from MSB to LSB is the same asthe eighth embodiment. In the preferred ninth embodiment, the respectivebit planes are sequentially entropy encoded one by one from the MSB tothe LSB. So, each entropy encoding block is independent. On the otherhand, in the preferred ninth embodiment, encoding is performed astridethe bit planes of the totality of the entropy encoding blocks existingin the one and the same sub-band. Referring to FIG. 20, the operation isexplained specifically.

FIG. 20 shows a cross-section (one-dimensional) of a bit plane of anentropy encoding block in a sub-band. In FIG. 20, the size of thesub-band block is denoted W_(S) and the entropy encoding size block isdenoted W_(E), indicating that the block of the shaded area has beenshifted up by two bits. The shifting-up operation newly generates a bitplane in a netted area. The result is that, if the totality of theentropy encoding blocks present in the same sub-band are entropy encodedon the bit plane basis from the MSB (bit plane Bp1 in FIG. 20) to theLSB (bit plane Bp2), the bit array becomes different from that in theinitial bit plane, as a result of the aforementioned shifting operation,so that the sequence of the encoded bit planes differs from one entropyencoding block to another.

For example, after shifting the bit plane of the block divided intosub-bands by a pre-set number of bits, the totality of the entropyencoding blocks existing in the same sub-band may be sequentiallyencoded in a direction from the MSB to the LSB on the bit plane basis.

In the present embodiment, picture quality scalability decoding, inwhich the picture quality is gradually improved stepwise, with theresolution remaining constant, may be realized by sequentially decodingfrom the MSB towards the LSB on the decoder side. This is effective onthe Internet or on the network, such as radio network, where the networkspeed is limited.

The above-mentioned blocks, split into sub-bands, are inherentlygenerated on wavelet splitting the specified spatial picture areas of aninput image. The block information may be furnished by means foranalyzing the motion information in a block picture or fineness of thetexture from one spatial block area picture forming an image to another.

On splitting into respective bit planes, the bit planes may beclassified into those for absolute value data of 0 and 1 and those forplus and minus signs. It is therefore possible to realize a highcompression ratio by arithmetic coding of these absolute value data andthe codes.

Tenth Embodiment

This tenth embodiment represents the expansion of the above-describedninth embodiment. In this ninth embodiment, bit-plane encoding from theMSB to the LSB is made independently from band to band. However, in apicture, more energy is generally concentrated in low-range components.So, by placing more emphasis on low-range sub-bands than on high-rangesub-bands, a picture with superior subjective picture quality may beexpected to be furnished for the same bitrate.

Based on this concept, the preferred tenth embodiment arrays the encodedbitstreams of the entropy encoding blocks from the lowest range sub-bandwith the maximum number of splitting stages towards the highest rangesub-band with the least number of the splitting stages to improve thesubjective picture quality for the same bitrate. On the side decoder,scalability decoding can be realized, in which the resolution of adecoded picture is progressively improved as the encoded bitstreamgenerated using means of the present embodiment keeps on to be decoded.

Eleventh Embodiment

If, in the present eleventh embodiment, the input picture is aninterlaced picture, the aforementioned shift-up means is used for ablock in question decided to exhibit marked motion from the analysisinformation provided by the block picture analysis unit, in order toshift the bit plane of the block area in question.

The interlaced picture is explained with reference to FIG. 21. Thetelevision broadcast we are familiar with uses an interlaced pictureobtained on interlaced scanning, and is made up of odd and even fields.This is illustrated in FIG. 21, in which lines La1, La2, . . . denoteodd fields and lines Lb1, Lb2, . . . denote even fields.

The interlaced picture is made up of odd fields and even fields,alternated with each other on the line basis, as shown by a framestructure on the left hand side of FIG. 21. If this interlaced pictureis encoded in the state of the frame picture, the efficiently mayoccasionally be lowered. That is, if, in an interlaced frame picture,the object shows vigorous motion, deviation tends to be apparent betweenodd and even fields, as in the frame structure shown on the left side ofFIG. 21. This leads to the lowered efficiently in the downstream sideencoding. By dividing the frame structure into field structures shown onthe right side of FIG. 21, deviation between neighboring lines iseliminated to prevent the efficiently from being lowered.

Therefore, if the interlaced frame picture as such is to be encoded, itis advisable to raise the picture quality in the block. As the analysisinformation, the analysis information 106 from the block pictureanalysis unit 1, as already explained in connection with the firstembodiment, shown in FIG. 1, may be used.

Twelfth Embodiment

In ths twelfth embodiment, for a block for entropy encoding, decided bythe above-described analysis information to be a still area, theabove-mentioned shift-up means is used to shift the bit plane of theblock area for entropy encoding. This takes account of the fact that astill area is generally more noticeable to the human visual system suchthat picture quality deterioration in the still area is detectable morereadily. The picture quality may be improved by shift-up of the bitplane of the block area.

Thirteenth Embodiment

This thirteenth embodiment speeds up the encoding processing of eachtile picture by parallel processing in case of wavelet transformencoding following tile splitting. That is, since the tiles obtained onsplitting a picture can be encoded separately from one another, asexplained in connection with the above-described fourth embodiment,high-speed encoding can be realized if the respective tile pictures areprocessed in parallel. For example, in a LSI (large-scale integratedcircuit) or a computer system, carrying plural CPUs, it sufficers toallocate the encoding processing for a sole tile picture to each CPU andto perform the tile encoding sequentially.

Fourteenth Embodiment

The fourteenth embodiment of the present invention is now explained. Theprevious embodiments all relate to encoding means for still pictures.However, since a concatenation of still pictures represent a movingpicture, the encoding technique may obviously be applied to encoding ofmoving pictures. In this case, there are required means for dividing thecontinuous moving picture into respective frames. In general, the NTSCsignals use a configuration of converting analog signals into digitalsignals and storing the moving picture of digital signals in a buffer onthe frame basis. The respective frames, thus stored, may then be encodedby still picture encoding means discussed above.

It is similarly possible to buffer a certain plural number of frames ofthe pictures and to then proceed to wavelet transform encoding. However,in this case, the memory capacity required is increased.

Fifteenth Embodiment

When encoding the quantization coefficients from one entropy encodingblock to another, by means discussed above, there are as many bit planesof the entropy encoding blocks as the number of planes from the MSB tothe LSB. In the present fifteenth embodiment, the respective bit planesare resolved into plural sub-bit planes.

This is illustrated in FIG. 22, in which a bit plane corresponding tothe MSB is made up of three sub-bit planes SB1, SB2 and SB3. A lowerorder bit plane is made up of four sub-bit planes of SB1, SB2, SB3 andSB4. These sub-bit planes may be processed in respective encoding pathswhich may be set arbitrarily. Specifically, it is more efficient toexecute the encoding as statistical data for optimum arithmetic encodingare estimated from the distribution information of a data string made upof binary data of 0 and 1. To this end, such technique alreadypublicized in a number of extended abstracts may be used.

It is noted that three encoding paths are provided in the entropyencoding of the JPEG2000 standard. If the totality of three encodingpaths are used, the compression efficiency may be maximum, however, theprocessing volume or time is increased. The smaller the number of theencoding paths, the lower becomes the compression efficiency. However,the processing volume or time is diminished. With the number of theencoding paths equal to zero, entropy encoding is not performed, withthe processing volume being equal to zero, such that the originalquantization coefficients data are issued directly as output. Theseencoding paths are selected independently for the respective sub-bitplanes.

The manner in which the encoding paths of the sub-bit planes are optedand selected for each entropy encoding block of a matrix in which thesub-bit planes in question exist is hereinafter explained. The criteriumfor the selection means is explained in the embodiments which will beexplained subsequently.

Sixteenth Embodiment

In this sixteenth embodiment, the manner in which the encoding paths ofthe sub-bit planes opted and selected are varied from one bit plane toanother is explained. In this sixteenth embodiment, option and selectionare made such that all of the encoding paths of three sub-bit planesSB1, SB2 and SB3 present in the MSB bit plane in FIG. 22 are processed,whereas only two of four sub-bit planes SB1, SB2, SB3 and SB4 present inthe lower bit plane are processed, thereby elevating the encodingefficiency while suppressing picture deterioration.

The reason is that the MSB affects the picture quality more stronglythan the LSB, that is that the MSB is more significant coefficient thanthe LSB. Therefore, in reducing the number of encoding bits, it isadvisable to perform the processing for encoding paths of the sub-bitplanes SB1 to SB3 present in the MSB bit planes preferentially to omitone or more or all of the encoding paths of the sub-bit planes SB1 toSB4 present in the bit plane towards the LSB.

Seventeenth Embodiment

In this seventeenth embodiment, it is shown that the encoding paths ofthe sub-bit planes opted and selected become variable with the bandtypes of the sub-band comprehending the sub-bit planes.

FIG. 23, for example, shows respective sub-bands generated as a resultof executing the three stages of the wavelet transform. For example, ifthe band comprehending the bit plane of the sub-bit plane is LL-0(lowest band), the encoding paths of the sub-bit plane is processedpreferentially, whereas, if the band comprehending the bit plane of thesub-bit plane is HH-3, existing in a high range, the priority is madelowest such that part or all of the encoding paths are omitted.

The above-described operation is performed for each of the encodingblocks corresponding to the sub-band for the specified area of thepicture to realize picture quality control individually associated withthe specified area of the picture.

Eighteenth Embodiment

In this eighteenth embodiment, more bit planes are encoded for theentropy encoding block of the lower sub-band than those for the entropyencoding block of the higher sub-bands.

That is, in FIG. 23, all or a majority of the bit planes from the MSB tothe LSB of the entropy encoding blocks of the lower sub-band LL-0 arepreferentially encoded, whereas the number of bit planes of the entropyencoding blocks to be encoded is reduced to a smaller number or to zerotowards a higher sub-band.

In this manner, the priority is raised to encode more bit planes towardsa lower band side where the picture energy is concentrated, thusproviding a picture in which more emphasis is attached to the lowerfrequency. Moreover, by performing the above processing for eachencoding block corresponding to a sub-band corresponding in turn to aspecified area in a picture, it becomes possible to perform picturequality control individually associated with specified areas of thepicture.

Nineteenth Embodiment

In this nineteenth embodiment, more encoding paths of sub-bit planes areprocessed in the encoding of respective bit planes in the entropyencoding blocks of the lower sub-bands than in the entropy encodingblocks of the higher sub-bands.

That is, all or a majority of the encoding paths of sub-bit planespresent in the respective bit planes from the MSB to the LSB of FIG. 22are processed in the entropy encoding blocks present in the lowestsub-band LL-0, as shown in FIG. 23. Conversely, the number of theencoding paths of the sub-bit planes is decreased in the encoding of therespective bit planes in the direction of the increasing frequency, thatis towards the highest frequency band HH-3.

The present nineteenth embodiment may be combined with the technique inwhich emphasis is placed on the encoding paths towards the MSB asexplained in connection with the sixteenth embodiment, whereby finerpicture quality control may be realized. The above processing can beperformed from one encoding block corresponding to a sub-band for thespecified area of the picture to another to realize picture qualitycontrol individually associated with the specified area in the picture.

It is also possible to multiply the quantization coefficients in theembodiment of FIG. 15 with the weighting coefficients explained inconnection with the first embodiment. In this case, since thequantization coefficients of the sub-band for the crucial picture bandhave already been multiplied by the weighting coefficients explained inconnection with the first embodiment, the coefficient distribution isoffset towards the MSB side. So, even if the encoding paths of thesub-bit planes are controlled to be increased or decreased from one bitplane from the MSB to the LSB to another as explained in connection withthe sixteenth embodiment, the emphasis is placed on the MSB side, withthe result that the picture quality of the picture area is maintained.

In these fifteenth to nineteenth embodiments, means of increasing ordecreasing the number of the bit planes from the MSB to the LSB for eachentropy encoding block present in the sub-band of the specified picturearea, the number of encoding paths of the sub-bit planes for each bitplane from the MSB to the LSB or the encoding paths of the sub-bitplanes from the low range to the high range, may be selectively used torealize fine picture quality control to maintain high picture quality.

The illustrative application of the respective embodiment of the presentinvention may be enumerated by an electronic camera, a video camera, avideo codec (coder/decoder) for picture monitoring, codec for broadcastVTR, portable and mobile picture transmission/reception terminal (PDA),printer, satellite picture, codec for pictures for medical use, softwaremodules thereof, games, texture companding equipment used in athree-dimensional computer graphics and software modules thereof.

The present invention has been disclosed only by way of illustration andshould not be interpreted in a limiting fashion. On the contrary, thepresent invention can be modified within the scope interpreted in lightof the description of the following claims.

1. A picture encoding apparatus comprising: memory means for writing andstoring an input picture from one line to another; wavelet transformmeans for applying a wavelet transform in the horizontal and verticaldirections each time a number of lines required for the wavelettransform is stored in said memory means, the wavelet transform meansgenerating wavelet transform coefficients for a plurality of sub-bands;block picture analysis means for analyzing the input picture andgenerating block area analysis information; block area weight computingmeans for weighting a block area from the block area analysisinformation and outputting block area weights; scalar quantization meansfor scalar quantizing wavelet transform coefficients for each sub-bandobtained from said wavelet transform means and outputting scalarquantization coefficients; block area weight coefficient computing meansfor multiplying the block area weights with weighting coefficients froma table for each sub-band generated at the time of the wavelet transformand with the scalar quantization coefficients and outputting themultiplied result as quantization coefficients; and entropy encodingmeans for entropy encoding the quantized coefficients for each sub-bandfrom said quantization means in units of block areas when the number ofsamples of said quantized coefficients has reached the size of a blockarea required for entropy encoding.
 2. The picture encoding apparatusaccording to claim 1 wherein weighting coefficients of said table foreach sub-band are such that, the larger the number of sub-band splittingstages, the larger the weighting coefficients become and the higher thepriority placed on the weighting coefficients becomes, and conversely,the smaller the number of the splitting stages, the smaller theweighting coefficients become and the lower the priority placed on theweighting coefficients becomes, and such that, in sub-bands of the samesplitting stage, the weighting coefficients become smaller for the highrange than for the low range to decrease the priority of the weightingcoefficients.
 3. The picture encoding apparatus according to claim 1,wherein said entropy encoding means resolve quantization coefficients insaid block into bit planes composed of binary data and executesarithmetic encoding depending on the occurrence probability distributionof symbols in each bit plane, and wherein the estimation of saidprobability distribution is performed only on data in a predeterminedblock.
 4. The picture encoding apparatus according to claim 1 whereinsaid input picture is split into a plurality of rectangular tiles andwritten in said memory means.
 5. The picture encoding apparatusaccording to claim 1 wherein said input picture is a continuous pictureof a plurality of frames and wherein the input continuous picture issequentially encoded from one frame to another.
 6. A picture encodingmethod comprising: writing and storing an input picture in memory meansfrom one line to another; applying a wavelet transform in the horizontaland vertical directions each time a number of lines required for thewavelet transform is stored in said memory means to generate wavelettransform coefficients for a plurality of sub-bands; analyzing the inputpicture and generating block area analysis information; weighting ablock area from the block area analysis information and outputting blockarea weights; scalar quantizing the wavelet transform coefficients foreach sub-band and outputting scalar quantization coefficients;multiplying the block area weights with weighting coefficients from atable for each sub-band generated at the time of the wavelet transformand with the scalar quantization coefficients and outputting themultiplied result as quantization coefficients; and entropy encoding thequantized coefficients for each sub-band in units of block areas whenthe number of samples of said quantized coefficients has reached thesize of a block area required for entropy encoding.